Cross-Modal Message Passing for Two-stream Fusion

Processing and fusing information among multi-modal is a very useful technique for achieving high performance in many computer vision problems. In order to tackle multi-modal information more effectively, we introduce a novel framework for multi-modal fusion: Cross-modal Message Passing (CMMP). Specifically, we propose a cross-modal message passing mechanism to fuse two-stream network for action recognition, which composes of an appearance modal network (RGB image) and a motion modal (optical flow image) network. The objectives of individual networks in this framework are two-fold: a standard classification objective and a competing objective. The classification object ensures that each modal network predicts the true action category while the competing objective encourages each modal network to outperform the other one. We quantitatively show that the proposed CMMP fuses the traditional two-stream network more effectively, and outperforms all existing two-stream fusion method on UCF-101 and HMDB-51 datasets.Comment: 2018 IEEE International Conference on Acoustics, Speech and Signal Processin

Observe matter falling into a black hole

It has been well known that in the point of view of a distant observer, all in-falling matter to a black hole (BH) will be eventually stalled and "frozen" just outside the event horizon of the BH, although an in-falling observer will see the matter falling straight through the event horizon. Thus in this "frozen star" scenario, as distant observers, we could never observe matter falling into a BH, neither could we see any "real" BH other than primordial ones, since all other BHs are believed to be formed by matter falling towards singularity. Here we first obtain the exact solution for a pressureless mass shell around a pre-existing BH. The metrics inside and interior to the shell are all different from the Schwarzschild metric of the enclosed mass. The metric interior to the shell can be transformed to the Schwarzschild metric for a slower clock which is dependent of the location and mass of the shell. Another result is that there does not exist a singularity nor event horizon in the shell. Therefore the "frozen star" scenario is incorrect. We also show that for all practical astrophysical settings the in-falling time recorded by an external observer is sufficiently short that future astrophysical instruments may be able to follow the whole process of matter falling into BHs. The distant observer could not distinguish between a "real" BH and a "frozen star", until two such objects merge together. It has been proposed that electromagnetic waves will be produced when two "frozen stars" merge together, but not true when two "real" bare BHs merge together. However gravitational waves will be produced in both cases. Thus our solution is testable by future high sensitivity astronomical observations.Comment: 7 pages, 2 figures. Proceeding of the conference "Astrophysics of Compact Objects", 1-7 July, Huangshan, China. Abridged abstrac